Skip to content

ci: stop GitHub Actions failure spam + Driver/Native artifact grid#13

Merged
Snider merged 3 commits into
devfrom
ci/artifact-matrix
Jul 22, 2026
Merged

ci: stop GitHub Actions failure spam + Driver/Native artifact grid#13
Snider merged 3 commits into
devfrom
ci/artifact-matrix

Conversation

@Snider

@Snider Snider commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Every push to dev/main was failing both hosted workflows and emailing the owner. Two verified signatures, both fixed at the documented surface:

  • setup-go cache: go.sum lives in the modules (go/, cli/), not the repo root — every setup-go now carries cache-dependency-path.
  • Taskfile macOS-ism: sysctl -n hw.ncpu hardened to a portable chain with a literal floor (|| echo 4); Mac behaviour unchanged (getconf returns 32, same as sysctl).

The load-bearing finding: lem cannot build without cgo (DuckDB bindings are //go:build cgo) — there is no pure-Go cross-compile; every cell is a native build on a matching runner, and windows/aarch64 is not a cell (duckdb ships no lib for it).

Matrix reality-audit: hosted-macOS metal cell REMOVED (hosted runners are < macOS 26 — a plain build would ship a metallib-less, non-runnable binary); replaced by a DORMANT self-hosted lane behind vars.ENABLE_MACOS_METAL (never queues or fails until a macOS 26 runner is registered). cpu cells: macos {aarch64, x86_64}, linux {x86_64, aarch64}, windows x86_64 — native cgo everywhere.

Driver/Native grid (docs/release-artifacts.md): each cell builds once, packages twice —

  • Native {os}-{arch}-lem-{backend}-{version}.zip → binary lem (run locally)
  • Driver {os}-{arch}-lem-driver-{backend}-{version}.zip → binary lem-{backend} (deploy on a compute box, consumed over lem's API)

GitLab's homelab runner keeps the amd/cuda cells (it has the toolchain; .gitlab-ci.yml untouched — it is green). Rolling dev prerelease mechanism kept, names aligned to the grid.

Live-run verification on this PR + a workflow_dispatch of Build on this branch before merge.

🤖 Generated with Claude Code
Co-Authored-By: Virgil virgil@lethean.io

…ifact grid

The GitLab side is green; the failure emails were GitHub Actions. Two verified
signatures, each fixed with the documented built-in (config is code — no hand-rolling):

1. "Restore cache failed: Dependencies file is not found ... pattern: go.sum"
   The module root is go/ (and cli/), not the repo root. Added setup-go's
   cache-dependency-path input to every failing setup-go (ci.yml -> go/go.sum;
   build.yml -> go/go.sum + cli/go.sum). deps.yml left alone (it was passing).

2. `task: Command "sysctl -n hw.ncpu" failed` (ubuntu exit 1 / windows 127)
   go-task evaluates global sh-vars eagerly, so the NCPU var must succeed on every
   platform. Hardened the fallback chain with a literal floor:
   getconf ... || nproc ... || sysctl -n hw.ncpu 2>/dev/null || echo 4
   Mac path unchanged (getconf _NPROCESSORS_ONLN already returns the core count = 32).

build.yml rewritten to an HONEST matrix + the Driver/Native release grid:
- Probe finding: lem links DuckDB via cgo; duckdb-go-bindings is //go:build cgo, so
  CGO_ENABLED=0 fails for every target. There is NO pure-Go lem and NO free
  cross-compile — each cell is a native cgo build. duckdb supports darwin/{amd64,arm64},
  linux/{amd64,arm64}, windows/amd64 (NOT windows/arm64).
- GitHub now builds only the cpu cells it can build natively (macos aarch64 + x86_64,
  linux x86_64 + aarch64, windows x86_64); the dishonest hosted-metal cell is removed.
- Metal rides a self-hosted macOS 26 lane, dormant behind `if vars.ENABLE_MACOS_METAL`
  so it never queues or fails on a push; interim = built manually on the maintainer's Mac.
  Hosted runners are < macOS 26 and cannot compile the Metal 4 metallibs; a plain build
  would ship a non-runnable, metallib-less binary.
- Each cell is packaged twice from one build: Native {os}-{arch}-lem-{backend}-{ver}.zip
  (binary `lem`) and Driver {os}-{arch}-lem-driver-{backend}-{ver}.zip (binary
  `lem-{backend}`); rolling dev prerelease kept, version aligned to dev-<short-sha>.
- Dropped GOWORK=off from the binary build: cli pins go/inference v0.14.0 which lags the
  local tree (missing the dataset pkg), so the workspace must wire cli -> local go/.

docs/release-artifacts.md settles the grid, the two packagings, per-cell CI ownership,
the metal trade-off, the cgo finding, and the not-yet-wired follow-ups (tagged GitHub
release; GitLab grid-named zips — left for on-box validation).

.gitlab-ci.yml untouched (green; gates own WHEN). Makefile untouched. cli/tui untouched.

Co-Authored-By: Virgil <virgil@lethean.io>
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 80286629-1ce6-4ebf-92b3-e5086c6f4496

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Snider and others added 2 commits July 22, 2026 15:07
Four failures, three platform-blind and one real fixture bug — all
reproduced and re-verified on the linux box (full portable suite
exit=0 there; agent/ai + driver 335 green on darwin):

- engine/metal gemm_steel_test.go had no build constraint — on linux
  the package collapsed to that one file and failed to build. Tagged
  darwin && arm64 like every sibling; linux now excludes the package
  cleanly.
- agent/ai TestReadEvents compared time.Time with != after a disk
  round-trip — passes only when the local zone makes time.Parse hand
  back time.Local (BST Macs), fails on UTC runners. Now .Equal.
- engine/hip: two contract tests carried pre-truth-pass expectations
  (standalone-native qwen3_6_moe, fitting 16GB plan). The truth-pass
  source (ArchitectureProfileNotes post-#50) is explicit: no native
  path replaced the retired composed detour. Realigned to
  metadata_only / Fits=false with the architecture gap reported
  honestly; the memory-plan maths assertions stay.
- engine/driver: fakeDriverScript ran a bare 'sleep' under the
  harness's deliberately EMPTIED PATH — exit 127 on spawn, every test
  driver an instant corpse, the crash-restart test racing its own
  restart storm (darwin's poll won the race; linux's never did).
  /bin/sleep by absolute path — the restart loop is now genuinely
  exercised on both platforms.

Co-Authored-By: Virgil <virgil@lethean.io>
…or pin

macos/x86_64: macos-13 runner label retired — queued forever, no hosted
Intel Mac remains. windows/x86_64: lem is not Windows-portable
(syscall.Kill + raw-fd reads outside constraints — three live compile
errors); the cell returns when a portability campaign lands, not
before. sdk: @openapitools/openapi-generator-cli npm wrapper is 2.x
(2.40.0) — 7.22.0 conflated it with the Java generator version it
manages; ETARGET on every run.

Live receipts: CI (portable suite) PASSED on this branch; cpu cells
macos/aarch64 + linux/x86_64 + linux/aarch64 built green; metal gate
skipped dormant as designed.

Co-Authored-By: Virgil <virgil@lethean.io>
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

💡 Need a hand with PR review? Try Gitar by Sonar!

- name: Install generators
run: |
npm install -g @openapitools/openapi-generator-cli@7.22.0
npm install -g @openapitools/openapi-generator-cli@2.40.0 # npm wrapper is 2.x; it manages the 7.x Java generator
@Snider
Snider merged commit f42f7b4 into dev Jul 22, 2026
9 of 10 checks passed
@Snider
Snider deleted the ci/artifact-matrix branch July 22, 2026 15:08
Snider added a commit that referenced this pull request Jul 22, 2026
Co-Authored-By: Virgil <virgil@lethean.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants